Information processing apparatus, information processing method, and program

ABSTRACT

There is provided an information processing apparatus for automatically generating information representing a context surrounding a user, the information processing apparatus including a recognition processing unit configured to perform, on the basis of user environment information including at least any of location information representing a location where a user is present, image information relating to an environment surrounding a user, and audio information relating to the environment, an analysis process of at least any of the location information, the image information, and the audio information included in the user environment information, at a predetermined time interval, and to recognize a context surrounding the user, using the acquired result of analysis relating to the user environment; and a context candidate information generating unit configured to generate context candidate information representing a candidate of the context surrounding the user, the context candidate information including, at least, information representing the context surrounding the user and information representing the user&#39;s emotion in the context, using the result of context recognition performed by the recognition processing unit.

This application is a continuation of U.S. application Ser. No.16/676,477, filed Nov. 7, 2019, which is a continuation of U.S.application Ser. No. 16/381,017, filed Apr. 11, 2019 (now U.S. Pat. No.10,853,650), which is a continuation of U.S. application Ser. No.15/303,391, filed Oct. 11, 2016 (now U.S. Pat. No. 10,311,303), which isbased on PCT filing PCT/JP2015/057861, filed Mar. 17, 2015, and claimspriority to Japanese Application No. 2014-106276, filed May 22, 2014,the entire contents of each are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatus,an information processing method, and a program.

BACKGROUND ART

In recent years, there is widely used Internet-based Social NetworkServices (SNS) or the like, as a tool for recording a person's ownactivity, or disclosing a person's own activity to a large number ofspecified or unspecified people.

Here, recording of a person's own activity to a social network serviceis performed by creating an article to be posted by the user himself orherself, attaching image data or the like to the article as necessary,and subsequently transmitting the data to a management server in chargeof managing the social network service. Accordingly, the user may bepreoccupied with the activity he or she is currently engaging and forgetto create the article to be posted.

In order to mitigate such a situation, Patent Literature 1 describedbelow, for example, discloses a method of automatically recognizing theuser's activity, and automatically generating an entertaining sentenceon the basis of the acquired recognition result.

CITATION LIST Patent Literature

Patent Literature 1: JP 2008-3655A

SUMMARY OF INVENTION Technical Problem

Using the method described in Patent Literature 1 allows an activitypattern such as “leisurely walking” or “restless movement” to beacquired as chronological data. However, the activity pattern acquiredby the aforementioned method mainly represents the user's movement orstate in a relatively short time. Therefore it is difficult to estimatea specific content of activity such “ate in a hotel restaurantyesterday” or “shopped in a department store today” from the history ofthe activity pattern. In addition, individual activities forming anactivity pattern acquired by the method described in Patent Literature 1are in themselves not performed by a purpose by the user, and thereforeit is unlikely that posting a sentence generated on the basis of theacquired activity pattern to a social network service helps to make thesentence seem interesting when reviewed at a later time.

On the other hand, an article posted to a social network service andstill seeming interesting is one relating to various contextssurrounding the user that have resulted from tangling of the user'sindividual activities in a complicated manner. Accordingly, in order tofurther improve the user's convenience with regard to posting to asocial network service, it is desired to realize a technique that allowsfor recognizing a context surrounding a user and automaticallygenerating information representing the context surrounding the user onthe basis of the result of context recognition.

Therefore, the present disclosure proposes an information processingapparatus, an information processing method, and a program capable ofautomatically generating information representing a context surroundinga user.

Solution to Problem

According to the present disclosure, there is provided an informationprocessing apparatus including: a recognition processing unit configuredto perform, on the basis of user environment information including atleast any of location information representing a location where a useris present, image information relating to an environment surrounding auser, and audio information relating to the environment, an analysisprocess of at least any of the location information, the imageinformation, and the audio information included in the user environmentinformation, at a predetermined time interval, and to recognize acontext surrounding the user, using the acquired result of analysisrelating to the user environment; and a context candidate informationgenerating unit configured to generate context candidate informationrepresenting a candidate of the context surrounding the user, thecontext candidate information including, at least, informationrepresenting the context surrounding the user and informationrepresenting the user's emotion in the context, using the result ofcontext recognition performed by the recognition processing unit.

According to the present disclosure, there is provided an informationprocessing method including: performing, on the basis of userenvironment information including at least any of location informationrepresenting a location where a user is present, image informationrelating to an environment surrounding a user, and audio informationrelating to the environment, an analysis process of at least any of thelocation information, the image information, and the audio informationincluded in the user environment information, at a predetermined timeinterval, and recognizing a context surrounding the user, using theacquired result of analysis relating to the user environment; andgenerating context candidate information representing a candidate of thecontext surrounding the user, the context candidate informationincluding, at least, information representing the context surroundingthe user and information representing the user's emotion in the context,using the result of context recognition.

According to the present disclosure, there is provided a program forcausing a computer to realize: a recognition processing function ofperforming, on the basis of user environment information including atleast any of location information representing a location where a useris present, image information relating to an environment surrounding auser, and audio information relating to the environment, an analysisprocess of at least any of the location information, the imageinformation, and the audio information included in the user environmentinformation, at a predetermined time interval, and recognizing a contextsurrounding the user, using the acquired result of analysis relating tothe user environment; and a context candidate information generatingfunction of generating context candidate information representing acandidate of the context surrounding the user, the context candidateinformation including, at least, information representing the contextsurrounding the user and information representing the user's emotion inthe context, using the result of context recognition performed by therecognition processing unit.

According to the present disclosure, a context surrounding a user isrecognized using a result of analysis performed for user environmentinformation with regard to a user environment, and context candidateinformation representing a candidate of the context surrounding theuser, including, at least, information representing the contextsurrounding the user and information representing the user's emotion inthe context is generated using the acquired context recognition result.

Advantageous Effects of Invention

According to the present disclosure as described above, it becomespossible to automatically generate information representing a contextsurrounding a user.

Note that the effects described above are not necessarily limitative.With or in the place of the above effects, there may be achieved any oneof the effects described in this specification or other effects that maybe grasped from this specification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an explanatory diagram illustrating an information processingsystem according to an embodiment of the present disclosure.

FIG. 2 is a block diagram illustrating an exemplary configuration of theinformation processing apparatus according to the embodiment.

FIG. 3 is a block diagram illustrating an exemplary configuration of arecognition processing unit included in the information processingapparatus according to the embodiment.

FIG. 4 is a block diagram illustrating an exemplary configuration of animage analysis unit included in the recognition processing unitaccording to the embodiment.

FIG. 5 is a block diagram illustrating an exemplary configuration of anaudio analysis unit included in the recognition processing unitaccording to the embodiment.

FIG. 6 is a block diagram illustrating an exemplary configuration of aplace/activity analysis unit included in the recognition processing unitaccording to the embodiment.

FIG. 7 is an explanatory diagram illustrating a place/activity analysisprocess performed by a recognition processing unit according to theembodiment.

FIG. 8 is an explanatory diagram illustrating a place/activity analysisprocess performed by a recognition processing unit according to theembodiment.

FIG. 9 is an explanatory diagram illustrating a place/activity analysisprocess performed by a recognition processing unit according to theembodiment.

FIG. 10 is an explanatory diagram illustrating a context recognitionprocess performed by the recognition processing unit according to theembodiment.

FIG. 11 is an explanatory diagram illustrating a context recognitionprocess performed by the recognition processing unit according to theembodiment.

FIG. 12A is an explanatory diagram illustrating a result of recognitionby the recognition processing unit according to the embodiment.

FIG. 12B is an explanatory diagram illustrating a result of recognitionby the recognition processing unit according to the embodiment.

FIG. 13 is an explanatory diagram illustrating a context candidateinformation generating process performed by a context candidateinformation generating unit included in the information processingapparatus according to the embodiment.

FIG. 14A is an explanatory diagram illustrating a context candidateinformation generating process performed by the context candidateinformation generating unit according to the embodiment.

FIG. 14B is an explanatory diagram illustrating a context candidateinformation generating process performed by the context candidateinformation generating unit according to the embodiment.

FIG. 15 is an explanatory diagram illustrating a context candidateinformation generating process performed by the context candidateinformation generating unit according to the embodiment.

FIG. 16 is an explanatory diagram illustrating a context candidateinformation generating process performed by the context candidateinformation generating unit according to the embodiment.

FIG. 17 is a block diagram illustrating an exemplary configuration of aninformation updating unit included in the information processingapparatus according to the embodiment.

FIG. 18 is a block diagram illustrating an exemplary configuration of anexternal device cooperation unit included in the information processingapparatus according to the embodiment.

FIG. 19 is an explanatory diagram illustrating an external devicecooperation process performed by the external device cooperation unitaccording to the embodiment.

FIG. 20A is an explanatory diagram illustrating an exemplary variationof the information processing apparatus according to the embodiment.

FIG. 20B is an explanatory diagram illustrating an exemplary variationof the information processing apparatus according to the embodiment.

FIG. 21 is an explanatory diagram illustrating an exemplary displayscreen of the information processing apparatus according to theembodiment.

FIG. 22A is an explanatory diagram illustrating an exemplary flow of acontext candidate information generating process in the informationprocessing apparatus according to the embodiment.

FIG. 22B is an explanatory diagram illustrating an exemplary flow of acontext candidate information generating process in the informationprocessing apparatus according to the embodiment.

FIG. 23 is a flowchart illustrating an exemplary flow of an informationprocessing method according to the embodiment.

FIG. 24 is a block diagram illustrating an exemplary hardwareconfiguration of the information processing apparatus according to anembodiment of the present disclosure.

DESCRIPTION OF EMBODIMENT(S)

Hereinafter, (a) preferred embodiment(s) of the present disclosure willbe described in detail with reference to the appended drawings. In thisspecification and the appended drawings, structural elements that havesubstantially the same function and structure are denoted with the samereference numerals, and repeated explanation of these structuralelements is omitted.

Note that description will be provided in the following order.

1. First embodiment

-   -   1.1. Information processing system    -   1.2. Information processing apparatus    -   1.3. Exemplary variation of information processing apparatus    -   1.4. Exemplary display screen    -   1.5. Exemplary flow of context candidate information generating        process    -   1.6. Information processing method

2. Hardware configuration of information processing apparatus

First Embodiment

<Information Processing System>

First, an information processing system including an informationprocessing apparatus according to a first embodiment of the presentdisclosure will be briefly described, referring to FIG. 1 . FIG. 1 is anexplanatory diagram schematically illustrating an overall configurationof an information processing system according to the present embodiment.

An information processing system according to the present embodimentincludes one or more information processing apparatuses 10 connected toa network 1 of various types such as the Internet, a wirelesscommunication network, and a mobile communication network. In theinformation processing system illustrated in FIG. 1 , the network 1 hasconnected thereto N information processing apparatuses 10A to 10N (mayalso be collectively referred to as “information processing apparatuses10”, in the following). Here, the number of information processingapparatuses 10 connected to the network 1 is not limited in particular.

The information processing system according to the present embodimenthas connected thereto, via the network 1, an information posting server3 that manages various information posting services of a social networksystem or the like. In addition, the network 1 has connected thereto aservice providing server 5 of various types such as, for example, aserver capable of acquiring information relating to locations, or aserver capable of acquiring information relating to the weather. Forexample, there are a GPS (Global Positioning System) server, a servermanaging a wireless communication network, a server managing a mobilecommunication network, as examples of a server capable of acquiringinformation relating to locations.

The information processing apparatus 10 according to the presentembodiment is capable of transmitting and receiving various informationto and from the information posting server 3, the service providingserver 5, or the like, via the network 1.

Here, the information processing apparatuses 10 according to the presentembodiment are not particularly limited in terms of types and may berealized using known devices. For example, the information processingapparatuses 10 may be personal digital assistants which users may carryaround such as mobile phones, smartphones, tablet type terminals,notebook PCs, or the like. In addition, the information processingapparatuses 10 may be various cameras such as digital cameras,camcorders, or the like. In addition, the information processingapparatuses 10 may be wearable terminals such as eye glasses,wristwatches, or various accessories.

In addition, the information processing apparatuses 10 according to thepresent embodiment are also capable of performing the processesdescribed below, cooperatively with an information processing server 20such as various computers or servers.

As thus, the configuration of the information processing systemaccording to the present embodiment has been briefly described above,referring to FIG. 1 .

<Information Processing Apparatus>

Next, an information processing apparatus 10 according to the presentembodiment will be described in detail, referring to FIGS. 2 to 19 .

First, a configuration of the information processing apparatus 10according to the present embodiment will be described, referring to FIG.2 . FIG. 2 is a block diagram schematically illustrating an exemplaryconfiguration of the information processing apparatus 10 according tothe present embodiment.

The information processing apparatus 10 according to the presentembodiment mainly includes, as illustrated in FIG. 2 , an informationacquisition unit 101, a recognition processing unit 103, a contextcandidate information generating unit 105, a display control unit 107, acontext information transmitting unit 109, an information updating unit111, an external device cooperation unit 113, and a storage unit 115.

Information Acquisition Unit 101

The information acquisition unit 101 is realized by, for example, a CPU(Central Processing Unit), a ROM (Read Only Memory), a RAM (RandomAccess Memory), an input device, a communication device, a sensor, andthe like. The information acquisition unit 101 acquires user environmentinformation including at least any of: location information representingthe location of a user carrying the information processing apparatus 10;image information relating to the environment surrounding the user;audio information relating to the environment surrounding the user, andthe like.

The information acquisition unit 101 acquires, at a predetermined timeinterval, various location information, various image information suchas still images or video images, or acquires audio information. Forexample, the information acquisition unit 101 may acquire information ona location where a user is present, using a GPS server, Wi-Fi, Cell ID,or the like. In addition, the information acquisition unit 101 mayacquire image information such as still images or video images, usingvarious cameras provided in the information processing apparatus 10. Inaddition, the information acquisition unit 101 may acquire audioinformation, using various microphones provided in the informationprocessing apparatus 10.

In addition, the information acquisition unit 101 may acquire, asbiological information intrinsic to a user, presence or absence of theuser's perspiration, the user's body temperature and heartbeat, biogenicsubstances existing on the surface or inside of the user's body, and thelike, using various biosensors provided in the information processingapparatus 10.

Note that the information acquisition unit 101 may acquire various userenvironment information by transmitting and receiving data to and fromvarious service providing servers 5 present on the network 1.

The various user environment information such as those acquired by theinformation acquisition unit 101 are output to the recognitionprocessing unit 103 described below whenever necessary, and used forvarious recognition processes performed by the recognition processingunit 103. In addition, the information acquisition unit 101 stores, forexample, the acquired location information, image information or audioinformation in the storage unit 115 or the like as attached data to beused for generating context candidate information described below, whenthe amount of characteristic characterizing the environment surroundingthe user according to a recognition process performed by the recognitionprocessing unit 103 described below satisfies a predetermined condition.

In addition, the various user environment information acquired by theinformation acquisition unit 101 are output to, and used as appropriatein, the context candidate information generating unit 105, the displaycontrol unit 107, the context information transmitting unit 109, theinformation updating unit 111, the external device cooperation unit 113,or the like, besides the recognition processing unit 103 and the storageunit 115 described below.

Recognition Processing Unit 103

The recognition processing unit 103 is realized by, for example, a CPU,a ROM, a RAM, a communication unit, and the like. The recognitionprocessing unit 103 performs, at a predetermined time interval, ananalysis process of at least any of location information, imageinformation and audio information included in the user environmentinformation, on the basis of user environment information acquired bythe information acquisition unit 101. In addition, the recognitionprocessing unit 103 performs a context recognition process ofrecognizing the context surrounding the user using the analysis resultrelating to the acquired user environment.

The recognition processing unit 103 has, as schematically illustrated inFIG. 3 , an image analysis unit 121, an audio analysis unit 123, aplace/activity analysis unit 125, and a context recognition unit 127,for example.

The image analysis unit 121 is realized by, for example, a CPU, a ROM, aRAM, a communication unit, and the like. The image analysis unit 121analyzes image information (i.e., image data) acquired by theinformation acquisition unit 101 to recognize the faces, landscapes,dishes, and various objects existing in the image corresponding to theimage data, or classify image scenes.

The image analysis unit 121 has, as illustrated in FIG. 4 , a faceanalysis unit 131, an image scene classification unit 133, a landscaperecognition unit 135, a dish recognition unit 137, and an objectrecognition unit 139, and the like.

The face analysis unit 131 is realized by, for example, a CPU, a ROM, aRAM, a communication unit, and the like. The face analysis unit 131 is aprocessing unit that recognizes whether or not a person's face isincluded in an image corresponding to the image data. An amount ofdetected face characteristic characterizing whether or not a face ispresent in the image is calculated by analyzing the image data by theface analysis unit 131 whenever necessary. When, in addition, a face ispresent in the image, the face analysis unit 131 identifies the regioncorresponding to the face, or identifies, for example, who the personcorresponding to the recognized face is. When the amount of detectedface characteristic calculated by the face analysis unit 131 is equal toor larger than a predetermined threshold value, the recognized imagedata is stored in the storage unit 115 or the like. Besides theforegoing, the face analysis unit 131 may identify the number of faces,coordinates, angles, or may calculate various properties such as facedetection score, face part location, presence or absence of smile,living thing, age, race, as the amount of detected face characteristic.

The image scene classification unit 133 is realized by, for example, aCPU, a ROM, a RAM, a communication unit, and the like. The image sceneclassification unit 133 is a processing unit that classifies what typeof scene an image corresponding to the image data is. Analyzing theimage data by the image scene classification unit 133 whenever necessaryallows calculation of the amount of scene classification characteristicindicating what type of scene the image is to be classified. Inaddition, analyzing the image data by the image scene classificationunit 133 whenever necessary allows identification of the place (e.g.,workplace, restaurant, etc.) where the user exists, or the context(driving a car, watching TV, etc.). When the amount of sceneclassification characteristic calculated by the image sceneclassification unit 133 is equal to or larger than a predeterminedthreshold value, the recognized image data is stored in the storage unit115 or the like.

The landscape recognition unit 135 is realized by, for example, a CPU, aROM, a RAM, a communication unit, and the like. The landscaperecognition unit 135 is a processing unit that recognizes whether or notan image corresponding to the image data is a captured image of alandscape. Analyzing the image data by the landscape recognition unit135 whenever necessary allows calculation of the amount of detectedlandscape characteristic characterizing whether or not there exists alandscape in the image. When, in addition, there exists a landscape inthe image, the landscape recognition unit 135 identifies the regioncorresponding to the landscape, or identifies the place corresponding tothe recognized landscape. When the amount of detected landscapecharacteristic calculated by the landscape recognition unit 135 is equalto or larger than a predetermined threshold value, the recognized imagedata is stored in the storage unit 115 or the like. In addition, thelandscape recognition unit 135 may also calculate a score indicating thedegree of being a beautiful landscape as the amount of detectedlandscape characteristic.

The dish recognition unit 137 is realized by, for example, a CPU, a ROM,a RAM, a communication unit, and the like. The dish recognition unit 137is a processing unit that recognizes whether or not a part correspondingto a dish is included in the image corresponding to the image data.Analyzing the image data using the dish recognition unit 137 whenevernecessary allows calculation of the amount of cooking detectioncharacteristic characterizing whether or not there exists a dish in theimage. When, in addition, there exists a dish in the image, the dishrecognition unit 137 identifies the region corresponding to the dish, oridentifies the category of the recognized dish (e.g., rice, miso soup,curry and rice, pasta, cake, etc.). When the amount of dish detectioncharacteristic calculated by the dish recognition unit 137 is equal toor larger than predetermined threshold value, the recognized image datais stored in the storage unit 115 or the like.

The object recognition unit 139 is realized by, for example, a CPU, aROM, a RAM, a communication unit, and the like. The object recognitionunit 139 is a processing unit that recognizes various objects existingin the image corresponding to the image data. Analyzing the image datausing the object recognition unit 139 whenever necessary allowscalculation of the amount of object detection characteristiccharacterizing whether or not there exist various objects in the image.When, in addition, there exists an object in the image, the objectrecognition unit 139 identifies the region corresponding to the object,or identifies the type of the recognized object. When the amount ofobject detection characteristic calculated by the object recognitionunit 139 is equal to or larger than the predetermined threshold value,the recognized image data is stored in the storage unit 115 or the like.

The results of processes performed by respective processing unitsdescribed above are output to the information acquisition unit 101, theaudio analysis unit 123, the place/activity analysis unit 125, and thecontext recognition unit 127 whenever necessary.

Note that the detailed method of the various analysis/recognitionprocesses performed by respective processing units described above isnot limited in particular, and known methods such as that disclosed inJP 2010-191934A, for example, may be used. In addition, the imageanalysis unit 121 may perform the aforementioned image recognitionprocess, using the service providing server 5 connected on the network 1and configured to provide an image recognition service.

Returning to FIG. 3 again, the audio analysis unit 123 included in therecognition processing unit 103 according to the present embodiment willbe described.

The audio analysis unit 123 is realized by, for example, a CPU, a ROM, aRAM, a communication unit, and the like. The audio analysis unit 123analyzes audio information (i.e., audio data) acquired by theinformation acquisition unit 101 to classify the audio data, orrecognize the content of the audio data.

The audio analysis unit 123 further includes an audio classificationunit 141, an audio recognition unit 143, and the like, as illustrated inFIG. 5 .

The audio classification unit 141 is realized by, for example, a CPU, aROM, a RAM, a communication unit, and the like. The audio classificationunit 141 analyzes audio data acquired by the information acquisitionunit 101 and calculates the amount of characteristic relating to theaudio data. More specifically, the audio classification unit 141performs a process of classifying whether the audio data isconversation, laughter, cheering, plosive audio (onomatopoeia), whetherit is an audio expressing praise such as applause, music, or the like.The classification process of such audio data may be performed by, forexample, referring to a database for audio analysis preliminarily storedin the storage unit 115 or the like, or performing a program for audioanalysis or the like.

In addition, the audio classification unit 141 may also analyze theaudio volume of the audio data and output the audio volume in aquantified manner, for example. Additionally, in a case where the audiodata is one which has originated from a human being such as conversationor laughter, the audio classification unit 141 may analyze whether theperson originating the audio is a male, a female, or a child. Theanalyses may be performed by, for example, analyzing the amplitude ofthe corresponding wave pattern, the frequency of the audio data, or thelike, referring to the spectra representing the audio.

Note that, when the amount of various characteristics relating to audiodata calculated by the audio classification unit 141 (e.g., amount ofconversation detection characteristic, amount of laughter detectioncharacteristic, amount of cheering detection characteristic, amount ofonomatopoeia detection characteristic, amount of music detectioncharacteristic) has reached or exceeded a predetermined threshold value,the recognized audio data is stored in the storage unit 115 or the like.

The audio recognition unit 143 is realized by, for example, a CPU, aROM, a RAM, a communication unit, and the like. The audio recognitionunit 143 analyzes the audio data by a known audio recognition processand a known language recognition process, converts the content of theaudio data into text data, or recognizes the content of the audio dataitself. Accordingly, it is possible to identify the content of audiodata and various words included in the audio data.

The results of processes performed by respective processing unitsdescribed above are output to the information acquisition unit 101, theimage analysis unit 121, the place/activity analysis unit 125, and thecontext recognition unit 127 whenever necessary.

Note that the detailed method of the various analysis/recognitionprocesses performed by respective processing units described above isnot limited in particular, and known methods such as that disclosed inJP 2010-191934A, for example, may be used. In addition, the audioanalysis unit 123 may perform the aforementioned audio recognitionprocess, using the service providing server 5 connected on the network 1and configured to provide an audio recognition service.

Returning to FIG. 3 again, the place/activity analysis unit 125 includedin the recognition processing unit 103 according to the presentembodiment will be described.

The place/activity analysis unit 125 is realized by, for example, a CPU,a ROM, a RAM, a communication unit, and the like. The place/activityanalysis unit 125 analyzes the location information acquired by theinformation acquisition unit 101 or the information output from anacceleration sensor or the like to identify the place where the user ispresent, or the content of the activity being performed by the user.

The place/activity analysis unit 125 further includes, as illustrated inFIG. 6 , an activity recognition unit 151, a place informationacquisition unit 153, a weather information acquisition unit 155, andthe like.

The activity recognition unit 151 is realized by, for example, a CPU, aROM, a RAM, a communication unit, and the like. The activity recognitionunit 151 analyzes the location information acquired by the informationacquisition unit 101 or the information output from the accelerationsensor or the like to calculate the amount of characteristiccharacterizing the user's activity. In addition, the activityrecognition unit 151 further uses the calculated amount ofcharacteristic to recognize the content of the activity being performedby the user. Accordingly, the activity recognition unit 151 maydetermine whether the user is staying or moving, or determine thetransportation means used by the user to move around. As a result, theactivity recognition unit 151 may grasp the user's state such as whetherthe user is walking, running, resting, jumping, moving on an elevator,or moving on a car, a train or a bicycle.

An exemplary activity recognition process performed by the activityrecognition unit 151 will be briefly described, referring to FIG. 7 .FIG. 7 is an explanatory diagram schematically illustrating theexemplary activity analysis process.

The activity recognition unit 151 first uses motion sensor data such asacceleration/gyro to extract the amount of characteristic from sensordata by performing known signal processing such as calculating theaverage, variance, frequency filter response on such sensor data. Inaddition, the activity recognition unit 151 may recognize a person'smovement, posture, vehicle or the like, using known machinelearning/pattern recognition techniques such as boosting, neuralnetwork, Hidden Markov Model (HMM).

In addition, the activity recognition unit 151 may use the image dataand the audio data (or the result of analysis by the image analysis unit121 or the audio analysis unit 123) as illustrated in FIG. 7 in order tofurther improve the recognition precision.

Details of the aforementioned activity recognition process performed bythe activity recognition unit 151 are not limited in particular, andknown methods such as that disclosed in JP 2010-198595A, for example,may be used. In addition, the activity recognition unit 151 may performthe aforementioned activity recognition process, using the serviceproviding server 5 connected on the network 1 and configured to providean activity recognition service.

The place information acquisition unit 153 is realized by, for example,a CPU, a ROM, a RAM, a communication unit, and the like. The placeinformation acquisition unit 153 analyzes the location informationacquired by the information acquisition unit 101 or the informationoutput from a barometric sensor or the like to acquire place informationrepresenting the place where a user is present, using the serviceproviding server 5 or the like that provides a place acquisition serviceas necessary. In the following, a place information acquisition processperformed by the place information acquisition unit 153 will bespecifically described, referring to FIGS. 8 and 9 . FIGS. 8 and 9 areexplanatory diagrams illustrating the place information acquisitionprocess.

The place information acquisition unit 153 searches a place databasestored in the storage unit 115 or the like (or various servers existingon the network 1), on the basis of the information representing thelatitude and the longitude among the place information acquired by theinformation acquisition unit 101. It is preferable that there exist inthe place database, as illustrated in FIG. 8 , a private place database(abbreviated as “DB”, in the following) storing private places such ashouses, workplaces and schools, and a public place DB storing publicplaces such as restaurants, coffee shops, stations, stores and parks.

Next, the place information acquisition unit 153 generates a placecandidate list illustrated in FIG. 8 , within a range (latitude andlongitude of the current location+a radius of several to tens of meters)according to a latitudinal or longitudinal error, referring to theprivate place DB and the public place DB. As illustrated in FIG. 8 , thedata representing respective places has names and categories of theplaces associated with each other.

When generating such a place candidate list, the place informationacquisition unit 153 may determine the altitude (floor of a building orthe like, to be more specific) to narrow down the place candidate list,using the output from the barometric sensor or the like.

The place candidate list generated in the aforementioned manner isusually not uniquely determined in the urban area and may includeseveral to tens of place candidates, as illustrated in FIG. 8 .

Next, the place information acquisition unit 153 uses the image data andthe audio data (or the result of analysis by the image analysis unit 121and the audio analysis unit 123) as illustrated in FIG. 9 in order torecognize and define the place.

For example, the place information acquisition unit 153, using atechnique for classifying scenes from the image data (e.g., generating adiscriminator from the image data preliminarily collected in largeamounts using machine learning), classifies scenes of restaurants,coffee shops, stations, stores, parks, houses, workplaces, schools andthe like, and calculates scores of respective scenes. On this occasion,the image scene classification may use depth information as the amountof characteristic besides color/brightness. In addition, the placeinformation acquisition unit 153 classifies audio scenes such asconversation, laughter and music, using a technique for classifyingaudio (e.g., preliminarily collecting samples and generating adiscriminator by machine learning), and calculates scores of respectiveaudio scenes.

Next, the place information acquisition unit 153 inputs the image sceneclassification scores and the audio scene classification scores to aplace category discriminator. Such a place category discriminatordiscriminates the place category to be a “restaurant” on the basis of apreliminarily and statistically learned result when, for example, theimage scene classification score of “restaurant” and the audioclassification score of “conversation” are both high.

Note that the place information acquisition unit 153 may cooperate theimage analysis unit 121 and the audio analysis unit 123 performing asimilar process to use the result of analysis by the image analysis unit121 and the audio analysis unit 123 as appropriate, without performing aprocess that uses the discriminator as described above.

Using the discrimination result of the place category acquired asdescribed above, the place information acquisition unit 153 sorts thegenerated place candidate list. In the example illustrated in FIG. 9 ,the discrimination result of the place changes from “workplace” to “XXcoffee shop”.

Performing the process described above allows the place informationacquisition unit 153 to acquire the place information representing theplace where the user is present.

Returning to FIG. 6 again, the weather information acquisition unit 155will be briefly described.

The weather information acquisition unit 155 is realized by, forexample, a CPU, a ROM, a RAM, a communication unit, and the like. Theweather information acquisition unit 155 acquires various weatherinformation (meteorological information) such as weather, highesttemperature, lowest temperature, probability of precipitation, windvelocity, humidity, atmospheric pressure of the place where the user ispresent, using the service providing server 5 (e.g., a weather forecastproviding server) or the like providing a weather informationacquisition service. Using the weather information acquired in thismanner, the place/activity analysis unit 125 may increase the precisionof the analysis process to be performed.

The results of processes performed by respective processing unitsdescribed above are output to the information acquisition unit 101, theimage analysis unit 121, the audio analysis unit 123, and the contextrecognition unit 127 whenever necessary.

Returning to FIG. 3 again, the context recognition unit 127 included inthe recognition processing unit 103 according to the present embodimentwill be described.

The context recognition unit 127 is realized by, for example, a CPU, aROM, a RAM, a communication unit, and the like. The context recognitionunit 127 uses various analysis results relating to the user environmentacquired by the image analysis unit 121, the audio analysis unit 123,the place/activity analysis unit 125, or the like, to recognize thecontext surrounding the user.

Here, “context surrounding the user” refers to the user's life-levelactivity estimated by performing a further recognition process on thebasis of the result of analysis relating to “time”, “place” and“activity”, with the “time”, “place” and “activity” being combined in ahybrid manner. As an example of such a context, there may be mentionedeating, shopping, working, meeting a person, travelling, playing,sports, moving, housework, appreciating artworks, relaxing, sleeping,etc.

Specifically, the context recognition unit 127 recognizes the contextsurrounding the user by using the results of image analysis, audioanalysis, time analysis, place analysis, and activity analysis, asschematically illustrated in FIG. 10 , and applying methods such as arule-based process, the time-series pattern recognition process, or thelike. In addition, the context recognition unit 127 outputs informationrepresenting the result of context recognition to the context candidateinformation generating unit 105 described below each time the contextsurrounding the user changes.

Here, the information relating to time, among the time analysis resultsillustrated in FIG. 10 , may be acquired by referring to the timeinformation held by the information processing apparatus 10. Inaddition, the context recognition unit 127 may use such time informationand general prior knowledge to determine what the context of the currenttime is like. For example, when a day of interest falls on Monday toFriday, it is possible to identify the day of interest as a weekday, oras a weekend when the day of interest is either Saturday or Sunday. Inaddition, it is determined to be morning when, for example, the time ofday in question lies between 6 and 11 o'clock, determined to beafternoon when the time of day in question lies between 12 and 15o'clock, determined to be evening when the time of day in question liesbetween 16 and 18 o'clock, and determined to be night when the time ofday in question lies between 19 and 5 o'clock of the next day.

Furthermore, the context recognition unit 127 may also identify theuser's usual activity pattern on the basis of the user's activityhistory and determine the context of the time of day specific to theuser. For example, it is possible to determine, on the basis of theusual activity pattern, the time zone of 8 to 9 o'clock on Monday toFriday to be commuting time, the time zone of 9 to 18 o'clock on Mondayto Friday to be work time, and the time zone of 20 to 21 o'clock onSaturday to be dinner time.

In addition, the result of image analysis illustrated in FIG. 10 may usethe result of analysis performed by the image analysis unit 121, and theresult of audio analysis illustrated in FIG. 10 may use the result ofanalysis performed by the audio analysis unit 123. In addition, theresults of place analysis and activity analysis illustrated in FIG. 10may use the result of analysis performed by the place/activity analysisunit 125.

Here, the rule-based process, among the context recognition processesperformed by the context recognition unit 127, is a process of applyingthe IF-THEN rule to respective analysis results relating to “time”,“place” and “activity”, and determining a context corresponding to therule having satisfied the condition to be the context surrounding theuser. For example, it may be determined that, on the basis of therule-based process, “the user is ‘working’ when the user is in theuser's workplace and sitting during work time”, “the user is ‘eating’when the user is in a restaurant around noon and a dish exists in theimage data”, “the user is ‘shopping’ when the user is walking around ina supermarket on the way home”, “the user is ‘travelling’ when the useris at a place far away from his workplace”.

In addition, the time-series pattern recognition process, among thecontext recognition processes performed by the context recognition unit127, is a type of machine learning technology such as the Hidden MarkovModel method, which is a technique suitable for modeling of temporalpatterns. The process is an approach of recognizing the contextsurrounding the user by preliminarily teaching a stochastic modelcharacterizing each context using a large amount of learning data andsubstituting, into a pre-constructed stochastic model, the input data tothe context recognition unit 127.

Using the aforementioned approach, the context recognition unit 127determines that the user is “eating” on the basis of the stochasticmodel, when the result of image scene classification transitions in theorder of “restaurant→cooking→face” and the result of activityrecognition transitions in the order of “resting→resting→resting”, asillustrated in FIG. 11 for example. When, in addition, the result ofimage scene classification transitions in the order of“store→book→store” and the result of activity recognition transitions inthe order of “walking→resting→walking”, the context recognition unit 127determines that the user is “shopping” on the basis of the stochasticmodel.

An exemplary context recognition result acquired in the aforementionedmanner is illustrated in FIGS. 12A and 12B. As illustrated in FIGS. 12Aand 12B, various recognition results acquired by the recognitionprocessing unit 103 include respective results of analysis processesperformed by the image analysis unit 121, the audio analysis unit 123and the place/activity analysis unit 125, and results of contextrecognition acquired by combining the results of analysis processes(i.e., high-level context).

The information representing the result of context recognitionillustrated in FIGS. 12A and 12B is output from the context recognitionunit 127 to the context candidate information generating unit 105whenever necessary.

Context Candidate Information Generating Unit 105

Returning to FIG. 2 again, the context candidate information generatingunit 105 according to the present embodiment will be described indetail.

The context candidate information generating unit 105 uses the result ofcontext recognition performed by the recognition processing unit 103 togenerate context candidate information representing a candidate of thecontext surrounding the user including at least information representingthe context surrounding the user and information representing the user'semotion in the context. In the following, a context candidateinformation generating process performed by the context candidateinformation generating unit 105 will be specifically described,referring to FIGS. 13 to 16 .

The context candidate information generated by the context candidateinformation generating unit 105 includes at least text data formed bytext data representing the context and text data representing theemotion, as also illustrated in FIG. 13 . The context candidateinformation generating unit 105 according to the present embodiment isable to generate an expressive and natural sentence by adding anexpression representing an emotion.

In addition, the context candidate information may also include imagedata and audio data used for the analyses, as also illustrated in FIG.13 . In addition, various data representing the context surrounding theuser may also be attached, besides text data, image data and audio data.

Here, the text data automatically generated by the context candidateinformation generating unit 105 to represent the context is formed by asentence like “do ‘what’ with ‘whom’, ‘when’, ‘where’, and ‘how’”, asillustrated in FIG. 14A. Here, the information acquired from the resultof time recognition is applied to the part expressing “when” in the textdata representing the context, and the information acquired from theresult of place recognition is applied to the part expressing “where”.In addition, the information acquired from the results of facerecognition and audio recognition is applied to the part expressing“who” in the text data representing the context, and the informationacquired from the result of image analysis is applied to the partexpressing “what”. Furthermore, not only the information acquired fromthe result of activity recognition but also the information acquiredfrom the result of context recognition subsequently performed in ahybrid manner is applied to the part expressing “how” in the text datarepresenting the context.

Note that, when generating a sentence intended to be posted to theinformation posting server 3, the result of time recognitioncorresponding to “when” is not usually very important. Accordingly,whether or not to apply the result of time recognition corresponding to“when” in a sentence may be determined as appropriate. However, theremay be a case where focusing on the result of time recognition allowsfor grasping whether the recognized time is before, in the middle of, orafter a certain activity. In such a case, the context candidateinformation generating unit 105 may use the result of time recognitionso as to appropriately select the tense (i.e., present, past, future,perfect, etc.) of the sentence to be automatically generated.

A case may also arise where no analysis result corresponding to theso-called 5W1H, namely, “when”, “where”, “who”, “what” and “how” existsin the results of analysis and context recognition of the userenvironment output from the recognition processing unit 103. Therefore,it suffices that the context candidate information generating unit 105corrects the sentence to be automatically generated as appropriate so asto generate a natural sentence, in a case where a part of the analysisresults is not acquired as illustrated in FIG. 14B.

On the other hand, the information representing an emotion according tothe present embodiment is generated by simplifying and expressing theemotion felt by the user into N types and switching sentence candidatesaccording to the context. Accordingly, the context candidate informationgenerating unit 105 generates text data representing an emotion using anemotion representation table illustrated in FIG. 15 . In the emotionrepresentation table illustrated in FIG. 15 , the column direction ofthe table corresponds to the emotion simplified into N types (threetypes in FIG. 15 ), and the row direction of the table corresponds toresults of context recognition.

The extent to which the emotion is simplified is not limited inparticular, and may be set as appropriate, as illustrated in FIG. 16 ,for example. FIG. 16 illustrates a case where the degrees of emotion areclassified into N types and the types of emotion are classified into Ntypes.

Simplifying the emotion into N types as illustrated in FIG. 16 allowsfor appropriately generating a sentence reflecting the user's emotion atthe time, for example, on the basis of a minimum user input such asselecting N types of buttons by the user, or emotion sensing usingbiosensors such as perspiration, heartbeat or temperature sensors.

In addition, switching sentence candidates representing the emotion inaccordance with the context allows for selecting the optimal expressionsuited for the context, making it possible to generate a naturalsentence. Furthermore, as illustrated in FIG. 15 , assigning a pluralityof sentence candidates to a single context allows for increasing thevariation of sentences. Increased variation of sentences also allows forgenerating a sentence expected to prevent the user from being bored.

Such an emotion representation table may be preliminarily prepared byknown methods. In addition, the emotion representation table may bepersonalized using sentences posted to a social network service by theuser, or the user's remarks.

Using the aforementioned method, the context candidate informationgenerating unit 105 generates context candidate information on the basisof the result of context recognition and the emotion representationtable, each time the context surrounding the user changes. Subsequently,the context candidate information generating unit 105 outputs thegenerated context candidate information to the display control unit 107for display to the user.

Display Control Unit 107

Returning to FIG. 2 again, the display control unit 107 included in theinformation processing apparatus 10 according to the present embodimentwill be described.

The display control unit 107 is realized by, for example, a CPU, a ROM,a RAM, an output device, a communication unit, and the like. The displaycontrol unit 107 performs display control when displaying variousprocessing results including context candidate information output fromthe context candidate information generating unit 105 on an outputdevice such as a display included in the information processingapparatus 10 or an output device provided outside the informationprocessing apparatus 10, or the like. Accordingly, the recognitionresult performed by the recognition processing unit 103 and the contextcandidate information generated by the context candidate informationgenerating unit 105 will be displayed on a predetermined area of thedisplay screen subjected to display control by the display control unit107. The user of the information processing apparatus 10 may grasp, onthe spot, various processing results such as context candidateinformation automatically generated by the information processingapparatus 10.

Context Information Transmitting Unit 109

The context information transmitting unit 109 is realized by, forexample, a CPU, a ROM, a RAM, a communication unit, and the like. Thecontext information transmitting unit 109 transmits the pieces ofinformation selected by the user from the context candidate informationgenerated by the context candidate information generating unit 105 tothe information posting server 3 providing a social network service, ascontext information representing the context surrounding the user.Accordingly, only the context information desired to be posted by theuser is posted to various social network services. As a result, postingbecomes easier for the user without having to prepare informationrelating to the context surrounding the user by himself or herself.

Information Updating Unit 111

The information updating unit 111 is realized by, for example, a CPU, aROM, a RAM, a communication unit, and the like. The information updatingunit 111 updates the expression representing the user's emotion includedin emotion representation table, using at least any of: the result ofanalysis relating to the user environment by the recognition processingunit 103; remarks or sentence expressions provided by the user; and theoutput from the sensor provided in the information processing apparatus10. In the following, an exemplary configuration of the informationupdating unit 111 according to the present embodiment will be brieflydescribed, referring to FIG. 17 . FIG. 17 is a block diagramillustrating an exemplary configuration of the information updating unitincluded in the information processing apparatus according to thepresent embodiment.

The information processing apparatus 10 according to the presentembodiment customizes the emotion representation table used by thecontext candidate information generating unit 105 for generating textdata representing an emotion to an emotion representation tablepersonalized for the individual user, using at least the content of theuser's remarks. Accordingly, the content of the context candidateinformation automatically generated by the context candidate informationgenerating unit 105 is provided with reality as if the user has createdit by himself or herself.

The information updating unit 111 described above has at least a habitextraction unit 161 and an emotion representation personalization unit167, and more preferably, at least either a contextdetermination/context recognition unit 163 or an emotiondetermination/emotion recognition unit 165, as illustrated in FIG. 17 .

The habit extraction unit 161 is realized by, for example, a CPU, a ROM,a RAM, a communication unit, and the like. The habit extraction unit 161extracts the user's habit such as frequently used words, frequently usedphrases, dialect, refrains or the like, from the user's everyday remarksaccording to remarks or sentence expressions provided by the user (i.e.,the user's voice or texts converted from the user's voice, or sentencescreated by the user).

Here, audio data or conversation over a telephone acquired via amicrophone provided in the information processing apparatus 10 may beused as the user's voice to be used for habit extraction. In addition,emails created by the user, content posted to a social network service,or the like, may be used as the sentence expressions provided by theuser.

The method of extracting the user's habit from the aforementioned datais not limited in particular, and it suffices to use various statisticalprocesses, machine learning techniques, pattern recognition techniques,or the like, as appropriate.

The information relating to the user's habit extracted by the habitextraction unit 161 is output to the emotion representationpersonalization unit 167.

The context determination/context recognition unit 163 is realized by,for example, a CPU, a ROM, a RAM, a communication unit, and the like.The context determination/context recognition unit 163 uses remarks orsentence expressions provided by the user to determine the contextreferring to a dictionary of words appearing in each of preliminarilyregistered contexts, or uses the result of analysis relating to the userenvironment performed by the recognition processing unit 103 torecognize the context. Accordingly, the context determination/contextrecognition unit 163 may identify the type of context (e.g., eating,working, etc.) in which the user has provided the remark or sentenceexpression of interest. The context determination/context recognitionunit 163 outputs the information relating to the acquired context to theemotion representation personalization unit 167. Accordingly, theemotion representation personalization unit 167 described below maypersonalize the emotion according to the context acquired by the contextdetermination/context recognition unit 163, in addition to the user'shabit extracted by the habit extraction unit 161.

Note that the method of determining or recognizing the context is notlimited in particular, and any known methods may be used, or such adetermination/recognition process may be performed by cooperating withthe service providing server 5 connected to the network 1. In addition,the functions of the context determination/context recognition unit 163may be realized through cooperation with the recognition processing unit103.

The emotion determination/emotion recognition unit 165 is realized by,for example, a CPU, a ROM, a RAM, a communication unit, and the like.The emotion determination/emotion recognition unit 165 uses remarks orsentence expressions provided by the user to determine the emotionreferring to a dictionary of words appearing in each of preliminarilyregistered emotions, or uses the output from a sensor provided in theinformation processing apparatus 10 (e.g., output from a biosensorrelating to perspiration, temperature, heartbeat, etc.) to recognize theemotion. Accordingly, the emotion determination/emotion recognition unit165 may identify the type of emotion (e.g., happy, tired, etc.) when theremark or sentence expression of interest is provided. The emotiondetermination/emotion recognition unit 165 outputs acquired informationrelating to the emotion to the emotion representation personalizationunit 167. Accordingly, the emotion representation personalization unit167 described below may personalize the emotion according to the emotionacquired by the emotion determination/emotion recognition unit 165, inaddition to the user's habit extracted by the habit extraction unit 161.

Note that the method of determining or recognizing the emotion is notlimited in particular, and any known methods may be used, or such adetermination/recognition process may be performed by cooperating withthe service providing server 5 connected to the network 1.

The emotion representation personalization unit 167 is realized by, forexample, a CPU, a ROM, a RAM, and the like. The emotion representationpersonalization unit 167 further uses the information relating to theuser's habit output from the habit extraction unit 161, preferably theinformation relating to the context and emotion, to customize theemotion representation table stored in a database relating to theemotion representation table. Accordingly, when using only informationrelating to the user's habit output from the habit extraction unit 161,phrases in the entre emotion representation table may be customized onthe basis of a knowledge that “the user has a habit of putting the words‘you know’ at the end of the phrase”, for example. When, alternatively,the information relating to context and emotion is also used, phrases inthe entre emotion representation table may be customized on the basis ofknowledge that “the user frequently uses the expression ‘reeeallyyummy!’ when pleased while eating”.

External Device Cooperation Unit 113

Returning to FIG. 2 again, the external device cooperation unit 113included in the information processing apparatus 10 according to thepresent embodiment will be described.

The external device cooperation unit 113 is realized by, for example, aCPU, a ROM, a RAM, a communication unit, and the like. The externaldevice cooperation unit 113 cooperates with another informationprocessing apparatus 10 with which mutual communication is possible soas to improve the precision of, or share, the result of generation ofthe context candidate information performed by the context candidateinformation generating unit 105. In the following, an exemplaryconfiguration of the external device cooperation unit 113 according tothe present embodiment will be briefly described, referring to FIGS. 18and 19 . FIG. 18 is a block diagram illustrating an exemplaryconfiguration of the external device cooperation unit included in theinformation processing apparatus according to the present embodiment,and FIG. 19 is an explanatory diagram illustrating an external devicecooperation process performed by the external device cooperation unitaccording to the present embodiment.

The external device cooperation unit 113 has a grouping unit 171 and acooperation processing unit 173, as illustrated in FIG. 18 .

The grouping unit 171 is realized by, for example, a CPU, a ROM, a RAM,a communication unit, and the like. The grouping unit 171 uses theresult of face recognition performed by the recognition processing unit103 and an address book having described therein various informationrelating to the user's acquaintances and friends stored in the storageunit 115 of the information processing apparatus 10, or the like, togroup other information processing apparatuses 10 coexisting in a placewhere the information processing apparatus 10 is present. It is verylikely that a plurality of users carrying the information processingapparatuses 10 sharing the place will participate in the same eventtogether (e.g., participating in a meal meeting), and therefore groupingthe information processing apparatuses 10 allows for efficientlyimproving the precision of, or sharing the result of generation of thecontext candidate information generated by the information processingapparatus 10.

The cooperation processing unit 173 is realized by, for example, a CPU,a ROM, a RAM, a communication unit, and the like. The cooperationprocessing unit 173 improves the precision of, or share the contextcandidate information among the plurality of information processingapparatuses 10 which have been grouped by the grouping unit 171.

For example, let us assume a context in which four users are eating. Insuch a case, the information processing apparatuses 10 carried by thefour users are supposed to capture images of the same dish content fromvarious directions, and therefore capable of sharing or improving theprecision of information by exchanging the information among the groupedinformation processing apparatuses 10.

Let us assume that, as illustrated in FIG. 19 for example, the devicescarried by people A and B have generated a context recognition resultthat they are “eating ‘Udon’”, and the device carried by a person C hasgenerated a context recognition result that he or she is “eating‘ramen’”, whereas the device carried by a person D has not performedcontext recognition for some reason. In such a case, cooperation of theexternal device cooperation units 113 of the respective informationprocessing apparatuses 10 with each other allows for improving theprecision of the recognition result performed by the device of theperson C, for example, to “eating ‘Udon’” on the basis of the principleof majority rule, or providing the device of the person D with themajority result of context recognition.

On this occasion, the cooperation processing unit 173 may not only sharetexts expressing the context but also may share image data, othervarious metadata, or the like.

Storage Unit 115

Returning to FIG. 2 again, the storage unit 115 included in theinformation processing apparatus 10 according to the present embodimentwill be described.

The storage unit 115 is realized by, for example, a RAM, a storagedevice, or the like included in the information processing apparatus 10according to the present embodiment. The storage unit 115 stores variouscontent data such as image data, audio data, or the like, generated bythe information processing apparatus 10. In addition, the storage unit115 has stored therein various object data to be displayed on thedisplay screen. The object data mentioned here includes, for example,any type of parts forming a graphical user interface (GUI) such asicons, buttons, thumbnails, or the like.

In addition, the storage unit 115 stores, whenever necessary, variousparameters or intermediate results of processes which have becomenecessary to be stored when the information processing apparatus 10according to the present embodiment performs a certain process, orvarious databases and programs, or the like. The storage unit 115 may befreely accessed for reading and writing of data by the informationacquisition unit 101, the recognition processing unit 103, the contextcandidate information generating unit 105, the display control unit 107,the context information transmitting unit 109, the information updatingunit 111, the external device cooperation unit 113, and the like.

An exemplary function of the information processing apparatus 10according to the present embodiment has been described above. Each ofthe aforementioned components may be configured using general-purposemembers or circuits, or may be configured by hardware specific to thefunction of each component. In addition, all the functions of respectivecomponents may be performed by the CPU or the like. Therefore, theconfiguration to be used may be changed as appropriate, according to thetechnical level at the time of implementing the present embodiment.

Note that it is possible to create, and load on a personal computer orthe like, computer programs for realizing respective functions of theinformation processing apparatuses according to the present embodimentdescribed above. In addition, it is possible to provide acomputer-readable storage medium storing such computer programs. Thestorage medium may be, for example, a magnetic disk, an optical disk, amagneto-optical disk, a flash memory, or the like. In addition, theaforementioned computer programs may be distributed via a network, forexample, without using a storage medium.

<Exemplary Variation of Information Processing Apparatus>

Next, an exemplary variation of the information processing apparatus 10according to the present embodiment described above will be brieflydescribed, referring to FIGS. 20A and 20B. FIGS. 20A and 20B areexplanatory diagrams illustrating an exemplary variation of theinformation processing apparatus according to the present embodiment.Note that FIGS. 20A and 20B illustrate only important parts ofrespective processing units included in the information processingapparatus 10 illustrated in FIG. 2 .

Although the foregoing description is given for case where theinformation processing apparatus 10 according to the present embodimentis realized in a single housing, respective processing units of theinformation processing apparatus 10 according to the present embodimentmay be distributed across a plurality of devices. In such a case, thefunctions of the information processing apparatus 10 are realized as anentire system through mutual cooperation of the plurality of devicesincluding respective processing units.

In the example illustrated in FIG. 20A, for example, only theinformation acquisition unit 101 is implemented in the informationprocessing apparatus 10, whereas the recognition processing unit 103 andthe context candidate information generating unit 105 are implemented inthe information processing server 20 connected to the network 1. In theexample illustrated in FIG. 20A, various information acquired by theinformation acquisition unit 101 of the information processing apparatus10 is transmitted to the information processing server 20 via thenetwork 1, and processing by the recognition processing unit 103 and thecontext candidate information generating unit 105 is to be performed bythe information processing server 20.

Additionally, in the example illustrated in FIG. 20B, the informationprocessing apparatus 10 has implemented therein the informationacquisition unit 101, as well as the image analysis unit 121, the audioanalysis unit 123 and the place/activity analysis unit 125 which areprocessing units, of the recognition processing unit 103, for analyzingthe user environment. In addition, the information processing server 20connected to the network 1 has implemented therein the contextrecognition unit 127 of the recognition processing unit 103 and thecontext candidate information generating unit 105. In the exampleillustrated in FIG. 20B, the information acquisition unit 101 of theinformation processing apparatus 10 acquires various information, andthe image analysis unit 121, the audio analysis unit 123, and theplace/activity analysis unit 125 analyze the acquired variousinformation to generate information representing the analysis result ofthe user environment. The information representing the analysis resultof the user environment is transmitted to the context recognition unit127 of the information processing server 20 and, after having beensubjected to the context recognition process, context candidateinformation is generated by the context candidate information generatingunit 105.

<Exemplary Display Screen>

Next, an exemplary display screen subject to display control by thedisplay control unit 107 of the information processing apparatus 10according to the present embodiment will be specifically described,referring to FIG. 21 . FIG. 21 is an explanatory diagram illustrating anexemplary display screen of the information processing apparatusaccording to the present embodiment.

The display screen controlled by the display control unit 107 displays,whenever necessary, an image captured by a camera or the like includedin the information processing apparatus 10. In addition, a part of thedisplay screen has provided thereon an area for displaying theamount-of-characteristic score relating to the user environment(amount-of-characteristic score display area), generated as a result ofexecution by the recognition processing unit 103, and an area fordisplaying the result of context recognition (recognition result displayarea), which are controlled by the display control unit 107. When, inaddition, a face exists in an image projected on to the display screen,a face detection frame is displayed on the part corresponding to theface, or a dish/object detection frame is displayed when there exists adish or an object. Furthermore, when context candidate information isgenerated by the context candidate information generating unit 105 onthe basis of the recognition result generated by the recognitionprocessing unit 103, the generated context candidate information isdisplayed on the context candidate information display area whenevernecessary.

The context candidate information is generated each time the contextsurrounding the user changes, and therefore a plurality of pieces ofgenerated context candidate information is displayed on the contextcandidate information display area. It is desired that the displaycontrol unit 107 divides the context candidate information display areainto layers or displays an object corresponding to a scroll bar, asillustrated in FIG. 21 , in order to indicate to the user that thereexists a plurality of pieces of generated context candidate information.

In addition, the display control unit 107 may prevent theamount-of-characteristic score display area and the recognition resultdisplay area from being perceived by the user.

<Exemplary Flow of Context Candidate Information Generating Process>

Next, an exemplary flow of context candidate information generatingprocess performed by the information processing apparatus 10 accordingto the present embodiment will be briefly described, referring to FIGS.22A and 22B. FIGS. 22A and 22B are explanatory diagrams illustrating anexemplary flow of the context candidate information generating processin the information processing apparatus according to the presentembodiment.

Upon activation of an application providing the aforementioned functionsin the information processing apparatus 10 according to the presentembodiment, a display screen illustrated in (a) of FIG. 22A is displayedto confirm which social network service the user is intending to postinformation. Upon selecting by the user an account to be used, apreparation is performed on the basis of the selection result toestablish connection with the information posting server 3 providing thecorresponding social network service.

Let us assume that the information processing apparatus 10 according tothe present embodiment is an accessory-type wearable terminal of aneye-glasses type or a button type, for example, born by the user. As theuser moves around, the information processing apparatus 10 keepsacquiring various image data and audio data. Here, it is assumed that,as illustrated in (b) of FIG. 22A, the location information from alocation acquisition sensor of the information processing apparatus 10has revealed that “the place where the user is present is XX Station”.In such a case, context candidate information indicating “at XX Stationnow” is displayed in the context candidate information display area ofthe display screen, and also a posting icon is displayed for starting aposting process to the information posting server 3. When the userselects the posting icon, the context candidate information beingdisplayed is transmitted to the information posting server 3 by thecontext information transmitting unit 109 as the context information.

Let us assume that a camera captures a certain person as illustrated in(c) of FIG. 22A at the next moment. In such a case, the recognitionprocessing unit 103 starts the recognition process on the basis of theacquired image information and, at the time point when the person isidentified, a face detection frame and the recognition result aredisplayed as illustrated in the drawing. Sequentially, upon completionof the context recognition result by the context recognition unit 127,context candidate information is generated, associating a text “at XXStation with Mr. A now” with the image data taken by the person A, asillustrated in (d) of FIG. 22A.

When, in addition, the person A disappears from the field of vision ofthe camera, context candidate information “at XX Station now” isgenerated in the context candidate information display area, asillustrated in (e) of FIG. 22B.

Let us assume that subsequently the user moves to enter a certain coffeeshop, and orders a cake. When the ordered cake is delivered and thus thecake exists in the field of vision of the camera, context candidateinformation “Eating a cake at XX Station now” is displayed, and also aposting icon is displayed for starting the posting process to theinformation posting server 3, as illustrated in (f) of FIG. 22B.Simultaneously, emotion selection icons are displayed on the displayscreen, with which the user selects his or her current emotion.

When an emotion selection icon is operated by the user, an expression“very delicious!”, which is a text expressing the emotion, is added asillustrated in (g) of FIG. 22B. When, in addition, the name of thecoffee shop is identified by the recognition processing unit 103, thedescription previously representing place of “XX Station” is changed to“XX cake shop” with a finer granularity. Furthermore, the image data ofthe cake is associated with text data “Eating a cake at XX cake shopnow. It's very delicious!”. The context candidate information isautomatically generated and accumulated in such a flow whenevernecessary.

In addition, the display control unit 107 may display a graphicalkeyboard on the display screen for changing the recognition result(e.g., the name “XX cake shop”, or the result of recognizing the object“cake”), allowing the recognition result to be changed.

<Information Processing Method>

Next, an exemplary flow of an information processing method according tothe present embodiment will be briefly described, referring to FIG. 23 .FIG. 23 is a flowchart illustrating the exemplary flow of theinformation processing method according to the present embodiment.

Upon activation of an application providing the aforementioned functionsin the information processing apparatus 10 according to the presentembodiment (step S101), user environment information such as locationinformation, image information, audio information, or the like isacquired whenever necessary and output to the recognition processingunit 103 by the information acquisition unit 101.

The recognition processing unit 103 analyzes the location information,image information, audio information, and the like acquired whenevernecessary (step S103) and, on the basis of the acquired analysis resultof the user environment, performs the context recognition process (stepS105). The result of context recognition generated by the contextrecognition unit 127 is output to the context candidate informationgenerating unit 105.

The context candidate information generating unit 105 generates thecontext candidate information, using at least the acquired contextrecognition result (step S107). The generated context candidateinformation is displayed on the display screen by the display controlunit 107 whenever necessary (step S109).

Here, the information processing apparatus 10 determines whether or nota posting operation has been performed by the user (step S111). When noposting operation has been performed by the user, the informationprocessing apparatus 10 returns to step S103 and continues the analysisof the user environment information acquired whenever necessary. When,on the other hand, a posting operation has been performed by the user,the context information transmitting unit 109 performs the postingprocesses by transmitting the user-selected context candidateinformation to the information posting server 3 as the contextinformation (step S113).

Subsequently, the information processing apparatus 10 determines whetheror not a termination operation of the application has been performed bythe user (step S115). When no termination operation has been performedby the user, the information processing apparatus 10 returns to stepS103 and continues the analysis of the user environment informationacquired whenever necessary. When, on the other hand, a terminationoperation has been performed by the user, the information processingapparatus 10 terminates the process.

A flow of the information processing method performed by the informationprocessing apparatus 10 according to the present embodiment has beenbriefly described above, referring to FIG. 23 .

(Hardware Configuration)

Next, the hardware configuration of the information processing apparatus10 according to the embodiment of the present disclosure will bedescribed in detail with reference to FIG. 24 . FIG. 24 is a blockdiagram for illustrating the hardware configuration of the informationprocessing apparatus 10 according to the embodiment of the presentdisclosure.

The information processing apparatus 10 mainly includes a CPU 901, a ROM903, and a RAM 905. Furthermore, the information processing apparatus 10also includes a host bus 907, a bridge 909, an external bus 911, aninterface 913, a sensor 914, an input device 915, an output device 917,a storage device 919, a drive 921, a connection port 923, and acommunication device 925.

The CPU 901 serves as an arithmetic processing apparatus and a controldevice, and controls the overall operation or a part of the operation ofthe information processing apparatus 10 according to various programsrecorded in the ROM 903, the RAM 905, the storage device 919, or aremovable recording medium 927. The ROM 903 stores programs, operationparameters, and the like used by the CPU 901. The RAM 905 primarilystores programs that the CPU 901 uses and parameters and the likevarying as appropriate during the execution of the programs. These areconnected with each other via the host bus 907 configured from aninternal bus such as a CPU bus or the like.

The host bus 907 is connected to the external bus 911 such as a PCI(Peripheral Component Interconnect/Interface) bus via the bridge 909.

The sensor 914 is a detecting means such as a sensor configured todetect the user's movement, or a sensor configured to acquire theinformation representing the current location. As an example of such asensor, there may be mentioned a motion sensor such as a three-axisacceleration sensor including an acceleration sensor, a gravitydetection sensor, a fall detection sensor or the like, or a three-axisgyro sensor including an angular velocity sensor, a shake correctionsensor, a geomagnetic sensor or the like, or a GPS sensor. In addition,the sensor 914 may be a detecting means configured to detectuser-specific biological information, or various information used toacquire such biological information. As an example of such a detectingmeans, there may be mentioned, for example, a sensor for detecting theuser's perspiration, a sensor for detecting the user's body temperatureand heartbeat, a sensor for detecting the biogenic substances existingon the surface or inside of the user's body. Furthermore, the sensor 914may include various measurement instruments besides those mentionedabove such as a thermometer, a photometer, a hygrometer or the like.

The input device 915 is an operation means operated by a user, such as amouse, a keyboard, a touch panel, buttons, a switch and a lever. Also,the input device 915 may be a remote control means (a so-called remotecontrol) using, for example, infrared light or other radio waves, or maybe an externally connected apparatus 929 such as a mobile phone or a PDAconforming to the operation of the information processing apparatus 10.Furthermore, the input device 915 generates an input signal based on,for example, information which is input by a user with the aboveoperation means, and is configured from an input control circuit foroutputting the input signal to the CPU 901. The user of the informationprocessing apparatus 10 can input various data to the informationprocessing apparatus 10 and can instruct the information processingapparatus 10 to perform processing by operating this input apparatus915.

The output device 917 is configured from a device capable of visually oraudibly notifying acquired information to a user. Examples of suchdevice include display devices such as a CRT display device, a liquidcrystal display device, a plasma display device, an EL display deviceand lamps, audio output devices such as a speaker and a headphone, aprinter, a mobile phone, a facsimile machine, and the like. For example,the output device 917 outputs a result obtained by various processingsperformed by the information processing apparatus 10. More specifically,the display device displays, in the form of texts or images, a resultobtained by various processes performed by the information processingapparatus 10. On the other hand, the audio output device converts anaudio signal such as reproduced audio data and sound data into an analogsignal, and outputs the analog signal.

The storage device 919 is a device for storing data configured as anexample of a storage unit of the information processing apparatus 10 andis used to store data. The storage device 919 is configured from, forexample, a magnetic storage device such as a HDD (Hard Disk Drive), asemiconductor storage device, an optical storage device, or amagneto-optical storage device. This storage device 919 stores programsto be executed by the CPU 901, various data, and various data obtainedfrom the outside.

The drive 921 is a reader/writer for recording medium, and is embeddedin the information processing apparatus 10 or attached externallythereto. The drive 921 reads information recorded in the attachedremovable recording medium 927 such as a magnetic disk, an optical disk,a magneto-optical disk, or a semiconductor memory, and outputs the readinformation to the RAM 905. Furthermore, the drive 921 can write in theattached removable recording medium 927 such as a magnetic disk, anoptical disk, a magneto-optical disk, or a semiconductor memory. Theremovable recording medium 927 is, for example, a DVD medium, an HD-DVDmedium, or a Blu-ray medium. The removable recording medium 927 may be aCompactFlash (CF; registered trademark), a flash memory, an SD memorycard (Secure Digital Memory Card), or the like. Alternatively, theremovable recording medium 927 may be, for example, an IC card(Integrated Circuit Card) equipped with a non-contact IC chip or anelectronic appliance.

The connection port 923 is a port for allowing devices to directlyconnect to the information processing apparatus 10. Examples of theconnection port 923 include a USB (Universal Serial Bus) port, anIEEE1394 port, a SCSI (Small Computer System Interface) port, and thelike. Other examples of the connection port 923 include an RS-232C port,an optical audio terminal, an HDMI (High-Definition MultimediaInterface) port, and the like. By the externally connected apparatus 929connecting to this connection port 923, the information processingapparatus 10 directly obtains various data from the externally connectedapparatus 929 and provides various data to the externally connectedapparatus 929.

The communication device 925 is a communication interface configuredfrom, for example, a communication device for connecting to acommunication network 931. The communication device 925 is, for example,a wired or wireless LAN (Local Area Network), Bluetooth (registeredtrademark), a communication card for WUSB (Wireless USB), or the like.Alternatively, the communication device 925 may be a router for opticalcommunication, a router for ADSL (Asymmetric Digital Subscriber Line), amodem for various communications, or the like. This communication device925 can transmit and receive signals and the like in accordance with apredetermined protocol such as TCP/IP on the Internet and with othercommunication devices, for example. The communication network 931connected to the communication device 925 is configured from a networkand the like, which is connected via wire or wirelessly, and may be, forexample, the Internet, a home LAN, infrared communication, radio wavecommunication, satellite communication, or the like.

Heretofore, an example of the hardware configuration capable ofrealizing the functions of the information processing apparatus 10according to the embodiment of the present disclosure has been shown.Each of the structural elements described above may be configured usinga general-purpose material, or may be configured from hardware dedicatedto the function of each structural element. Accordingly, the hardwareconfiguration to be used can be changed as appropriate according to thetechnical level at the time of carrying out the present embodiment.

The preferred embodiment(s) of the present disclosure has/have beendescribed above with reference to the accompanying drawings, whilst thepresent disclosure is not limited to the above examples. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present disclosure.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art based on the description of this specification.

Additionally, the present technology may also be configured as below.

(1)

An information processing apparatus including:

a recognition processing unit configured to perform, on the basis ofuser environment information including at least any of locationinformation representing a location where a user is present, imageinformation relating to an environment surrounding a user, and audioinformation relating to the environment, an analysis process of at leastany of the location information, the image information, and the audioinformation included in the user environment information, at apredetermined time interval, and to recognize a context surrounding theuser, using the acquired result of analysis relating to the userenvironment; and

a context candidate information generating unit configured to generatecontext candidate information representing a candidate of the contextsurrounding the user, the context candidate information including, atleast, information representing the context surrounding the user andinformation representing the user's emotion in the context, using theresult of context recognition performed by the recognition processingunit.

(2)

The information processing apparatus according to (1),

wherein the recognition processing unit outputs information representingthe result of context recognition to the context candidate informationgenerating unit each time the context surrounding the user changes.

(3)

The information processing apparatus according to (1) or (2),

wherein the information representing the user's emotion is generatedusing an emotion representation table preliminarily provided for each ofthe recognized contexts.

(4)

The information processing apparatus according to any one of (1) to (3),

wherein the context candidate information generating unit includes, inthe context candidate information, at least either the image informationor the audio information relating to the context surrounding the user.

(5)

The information processing apparatus according to any one of (1) to (4),further including:

a display control unit configured to display the result of recognitionperformed by the recognition processing unit and the context candidateinformation generated by the context candidate information generatingunit in a predetermined area of a predetermined display screen.

(6)

The information processing apparatus according to any one of (1) to (5),

wherein the recognition processing unit recognizes the context by atime-series pattern recognition process based on time course of theanalysis result, or a rule-based process based on predeterminedconditional processing, using the result of analysis relating to theuser environment.

(7)

The information processing apparatus according to any one of (1) to (6),further including:

an information updating unit configured to update expressionrepresenting the user's emotion included in an emotion representationtable preliminarily provided for each of the recognized contexts, usingat least any of the result of analysis relating to the user environmentperformed by the recognition processing unit, a remark or a sentenceexpression provided by the user, and an output from a sensor provided inthe information processing apparatus.

(8)

The information processing apparatus according to any one of (1) to (7),further including:

an external device cooperation unit configured to cooperate with anotherinformation processing apparatus with which mutual communication ispossible, and to improve precision of, or share, the result ofgeneration of the context candidate information performed by the contextcandidate information generating unit.

(9)

The information processing apparatus according to any one of (1) to (8),further including:

a context information transmitting unit configured to transmit a pieceof information selected by the user from the context candidateinformation generated by the context candidate information generatingunit to an information posting server providing a social networkservice, as context information representing the context surrounding theuser.

(10)

The information processing apparatus according to any one of (1) to (9),

wherein the information processing apparatus is a personal digitalassistant carried by a user, or a wearable terminal worn by a user.

(11)

An information processing method including:

performing, on the basis of user environment information including atleast any of location information representing a location where a useris present, image information relating to an environment surrounding auser, and audio information relating to the environment, an analysisprocess of at least any of the location information, the imageinformation, and the audio information included in the user environmentinformation, at a predetermined time interval, and recognizing a contextsurrounding the user, using the acquired result of analysis relating tothe user environment; and

generating context candidate information representing a candidate of thecontext surrounding the user, the context candidate informationincluding, at least, information representing the context surroundingthe user and information representing the user's emotion in the context,using the result of context recognition.

(12)

A program for causing a computer to realize:

a recognition processing function of performing, on the basis of userenvironment information including at least any of location informationrepresenting a location where a user is present, image informationrelating to an environment surrounding a user, and audio informationrelating to the environment, an analysis process of at least any of thelocation information, the image information, and the audio informationincluded in the user environment information, at a predetermined timeinterval, and recognizing a context surrounding the user, using theacquired result of analysis relating to the user environment; and

a context candidate information generating function of generatingcontext candidate information representing a candidate of the contextsurrounding the user, the context candidate information including, atleast, information representing the context surrounding the user andinformation representing the user's emotion in the context, using theresult of context recognition performed by the recognition processingunit.

REFERENCE SIGNS LIST

-   1 network-   3 information posting server-   5 service providing server-   10 information processing apparatus-   20 information processing server-   101 information acquisition unit-   103 recognition processing unit-   105 context candidate information generating unit-   107 display control unit-   109 context information transmitting unit-   111 information updating unit-   113 external device cooperation unit-   115 storage unit

The invention claimed is:
 1. A non-transitory, computer-readable mediumstoring instructions that, when executed by a processor on aninformation processing apparatus, control the information processingapparatus to implement a method comprising: receiving image informationread from image data storage; performing image recognition processes torecognize scene, object, and/or activity in the image information basedon a respective discriminator applied to the image information dependingon amount of characteristic of scene, object, and/or activityrespectively of the image information; outputting a score correspondingto the recognition as a result of performing the image recognitionprocesses; and generating context candidate information based on thescore and on the recognized scene, object, or activity in a hybrid formcomprising text data and at least one of audio data or image data,wherein the context candidate information represents the received imageinformation, and wherein the discriminators are generated from aplurality of images collected in advance and applied to a neuralnetwork.
 2. The non-transitory, computer-readable medium according toclaim 1, wherein the received image information is image informationcaptured by a camera and stored in the image data storage.
 3. Thenon-transitory, computer-readable medium according to claim 1, whereinthe object is at least one of a face, a landscape, or a dish.
 4. Thenon-transitory, computer-readable medium according to claim 3, wherein,if the object is recognized as a face, the method further comprisesoutputting at least one of a number of faces, coordinates, angles,presence or absence of smile, age, and race.
 5. The non-transitory,computer-readable medium according to claim 3, wherein, if the object isrecognized as a face, the method further comprises outputting a name ofa person belonging to the face.
 6. The non-transitory, computer-readablemedium according to claim 1, wherein the method further comprisestransmitting to a social network service an item of information selectedfrom the context candidate information.
 7. The non-transitory,computer-readable medium according to claim 1, wherein the methodfurther comprises generating the context candidate information each timean environment included in the image information changes.
 8. Thenon-transitory, computer-readable medium according to claim 1, whereinthe method further comprises displaying the context candidateinformation in a context candidate information area.
 9. Thenon-transitory, computer-readable medium according to claim 8, wherein aplurality of the context candidate information is displayed in thecontext candidate information area.
 10. The non-transitory,computer-readable medium according to claim 9, wherein the contextcandidate information includes text representing the context.
 11. Thenon-transitory, computer-readable medium according to claim 1, whereinthe method further comprises generating one or more classificationsbased on an amount of object characteristics detected in the imageinformation.
 12. The non-transitory, computer-readable medium accordingto claim 11, wherein the method further comprises: comparing the amountof object characteristics to a threshold for a particular object; andgenerating a classification for the particular object if the amount ofobject characteristics exceeds the threshold.
 13. A method fordetermining context candidate information from image information in aninformation processing apparatus comprising a processor, the methodcomprising: receiving image information read from image data storage inthe information processing apparatus; performing image recognitionprocesses to recognize scene, object, and/ or activity in the imageinformation based on a respective discriminator applied to the imageinformation depending on amount of characteristic of scene, object,and/or activity respectively of the image information; outputting ascore corresponding to the recognition as a result of performing theimage recognition processes; and generating context candidateinformation based on the score and on the recognized scene, object, oractivity in a hybrid form comprising text data and at least one of audiodata or image data, wherein the context candidate information representsthe received image information, and wherein the discriminators aregenerated from a plurality of images collected in advance and applied toa neural network.
 14. The method according to claim 13, wherein themethod further comprises transmitting to a social network service anitem of information selected from the context candidate information. 15.The method according to claim 13, wherein the method further comprisesgenerating the context candidate information each time an environmentincluded in the image information changes.
 16. An information processingapparatus comprising a processor configured to: receive imageinformation read from image data storage in the information processingapparatus; perform image recognition processes to recognize scene,object, and/or activity in the image information based on a respectivediscriminator applied to the image information depending on amount ofcharacteristic of scene, object, and/or activity respectively of theimage information; output a score corresponding to the recognition as aresult of performing the image recognition processes; and generatecontext candidate information based on the score and based on therecognized scene, object, or activity in a hybrid form comprising textdata and at least one of audio data or image data, wherein the contextcandidate information represents the received image information, andwherein the discriminators are generated from a plurality of imagescollected in advance and applied to a neural network.
 17. Theinformation processing apparatus according to claim 16, wherein thecircuitry is further configured to transmit to a social network servicean item of information selected from the context candidate information.